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'in predicting the performance gains to be expected from the intro duct ion of 
such aids. Further, in systems in which '-he responsibilities for some tasks 
are shared by the operator and an automated decision maker, these models 
might also be used within the system to coordinate the actions of the two 
decisiai makers. 

Senders [1] and Smallwood [2] have modeled human decision making in 
multiple process monitoring tasks. Senders postulated that the human monitor 
samples his displays in a manner which allows reconstruct ion of the displayed 
signals. An information theory approach is employed to determine how often 
and for what duration the human must sample each display. Smallwood proposed 
that the human operator forms an internal model of the processes he is 
monitoring and of the environment relevant to his task as a result of his 
past perceptions of them. A situation is considered in which the operator 
seeks to detect excursions of instruments beyond threshold values. The 
operator is modeled as directing his attention to the instrument whose 
current probability of exceeding threshold (based on the operator's internal 
model) is greatest. It might be noted, in passing, that the internal model 
concept discussed by Smallwood is perhaps as appropriate to the design of 
automated decision makers as it is to modeling the human decision maker. If 
the automated decision maker is to interact appropriately with the human, it 
would seem that its internal model of the relevant environment should include 
a model of the human. 

Carbonell [3,4] and Senders and Posner [5] have proposed queueing theory 
approaches to the modeling of human decision making in multiple process 
monitoring tasks. Carbonell uses a priority queueing discipline. He assumes 
that the human operator attempts to minimize the risk involved in not 
observing other instruments when he chooses to monitor a particular 
instrument. Senders and Posner employ a first come first served service 
discipline. They suggest two models which might be used to estimate the 
inter-observation intervals for an instrument (i.e., the time between 
arrivals of the instrument to the queue of instruments awaiting observation 
by the human monitor). The first model involves the degree of the observer's 
uncertainty about the value of the variable displayed on the instrument. The 
second model involves the probability that the displayed variable will exceed 
an acceptable limit. 

The models cited above emphasize the monitoring of displays, rather than 
the decisions or actions that result from the human operator's perception of 
the displayed values. The operator's motivation for monitoring the displays 
is the possibility that an event which requires his action will occur. The 
multi-task decision making problem addressed in this paper concerns the event 
detection and action selection decisions the operator makes on the basis of 
the information he gains through monitoring. 

Human decision making in suoh multi-task situations, then, might be 
modeled in terms of the manner in which the human detects events related to 
his tasks and the manner in which he allocates his attention among his tasks 
once he feels events have occurred. Gai and Curry [6] have developed a model 
of the human monitor in a failure detection task. The model has two stages, 
the first being a Kalman filter whioh estimates the states and observations 
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of . the monitored process and the second a decision mechanism which operates 
on the Kalman filter residuals using sequential analysis concepts. The model 
can be used to describe the human monitor's detection of additive failures in 
stationary random processes. 

Sheridan and Tulga [7] have modeled the manner in which the human 

operator allocates his attention among various tasks. They address a 

situation in which events present themselves unequivocally and use a dynamic 
programming approach to determine the action sequence which maximizes the 
operator's earnings. This action sequence is begun, but can be superceded by 
a new sequence calculated in response to the appearance of additional tasks. 

Rouse [8] has investigated the issue of allocation of decision making 
responsibility between a human operator and an automated decision maker. He 
presents a mathematical formulation of the multi-task decision making 

situation appropriate to the modeling of either decision maker. Eased on 

displayed information, the decision maker is assumed to generate 
probabilities that events have occurred in his tasks. He also generates 
density functions which characterize his perceptions of what might occur in 
his tasks while his attention is diverted to a particular task and how long 
his attention will be diverted should he decide to take a given action. 
Combining estimates of the probabilities events have occurred with the 
density functions of time between events in the tasks and action times with 
respect to the tasks, the decision maker chooses his actions to minimize an 
appropriate cost criterion. In this paper, we present a model of the human's 
event detection performance consistent with this mathematical formulation, 
describe an experimental study of event detection performance in a multiple 
process monitoring situation, and address the application of the model to the 
process monitoring situation. 


THE EVENT DETECTION MODEL 


The event detection model assumes that, in attempting to detect events, 
the human generates the probabilities that events have occurred. A 
discriminant analysis approach [9 >10] is used to model the human's generation 
of these probabilities. Our use of discriminant analysis to model the 
human's generation of event probabilities is motivated by the fact that this 
approach does not require explicit models of the systems the human is 
monitoring. An understanding of the systems is certainly helpful in 
determining the features to extract from the observations. But explicit 
models of the systems' structures are not required. 

For each task i, various features x^, j=1,2, ..., 1 ^, are extracted from 
the human's task related observations These features are properties of 

the observations that characterize (or are believed to characterize) the 
presence or absence of events related to the task. Following the extraction 
of a set of features, the value of a linear discriminant function 
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Yi s + . . . + v im^ x im^ 


( 1 > 


is calculated. Based on previous experience with the task, esti “ a * ea f\ 
made of the discriminant function coefficients v i<jt with whic 

to combine the feature values x^ to obtain the discrimirant functi^ 
v that best differentiates observations of events from the rest of the task 
related o^vaiis Estlaatea of the aean and variance of me d -r a nan 
function over observations of events and over the rest of the observation 
M Si fc a posteriori probability that an event »as »eourred la 
generated using the value of the disorlnlnant functlor. 

Of the means and variances of this score over events and “non-events , and an 
estimate of the a priori probability of the event. 

* If the human operator is forced to make a yes /no response on the 
presence of an event, we might assume that he chooses * e 
Maximizes his expected reward. We can then express his decision in a signal 
detection- manner and state that he should respond 'yes, an event related t 
task i has occurred” if the following inequality holds: 


P(ei/Yi) + Cp^ 

1 - P^/Yi) V H + C M 


( 2 ) 


P(e 4 /Yg) is the a posteriori probability that an event related to taskihas 
occurred. The value of this probability is generated by the eventdetection 
model. V rR is the value of correctly responding "no event 
rejection ) C , R C=.a is the cost of incorrectly responding event (a false 
alarm), V H is F the value of correctly responding "event” (a hit), and C M is 
the cost of incorrectly responding "no event" (a miss). 

It is predicted, then, that if the operator is forced to nakeayes/no 
decision on the presence of a task related event, he calculates the 
likelihood ratio of the event (the left hand side of Eq. (2)). He compar 
the magnitude of the likelihood ratio with a threshold det ermined by the 
values of correct responses and the costs of incorrect responses (the right 
hand side of Eq. (2)). He responds "event" if the likelihood ratio exceeds 

the threshold. 


THE EVENT DETECTION EXPERIMENT 


An experiment has been run employing a situation in which subjects 
simultaneously monitor several dynamic processes for the occurrence of events 
and make yes/no decisions on the presence of events in each process. Fi sure 
1 illustrates the display observed by the subjects in the experiment. T* 16 
Ititic iTplay Ts generated on a Tektronix 4010 by a time-shared DEC-System 
10 and depiots the measured values of toe outputs of nine processes ov «£ ™0 
sampling intervals (i.e., 101 points). The processes had identical second 
order system dynamics with a natural frequency of 0.75 rad/seo and a damp ng 
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Figure 1 . The Multiple Process Monitoring Situation 

ratio of 0.5* Samples were taken at 0.2 second intervals. The inputs to the 
processes were zero-mean Gaussian white noise sequences of identical 
variance. The displayed measurements were obtained by corrupting the process 
outputs with additive zero-mean Gaussian white noise sequences which normally 
had identical variance. The measurement noise variance was normally selected 
to yield measurements with signal-to-noise ratios of 25.0. An abnormal event 


in a process was defined by an increase in the measurement noise variance $ 

such that the signal-to-noise ratio following an event occurrence was ’ 

decreased to 95$ of the signal-to-noise ratio of the preceding measurement. j 

Thus, abnormal events became more pronounced with each measurement following ] 

their occurrence. j 

J 

5 

After scanning the nine process histories, the subject was given an f 


opportunity to key in the numbers of processes in which he had decided an j 

abnormal event had occurred. He was then given feedback regarding the actual j 

states of Hie processes he had keyed in ("1" indicating the normal state, "0" ! 

indicating the abnormal state). An iteration in a trial was completed by i 

erasing the display, scoring the subject's performance, and returning all ’ 

abnormal processes detected by the subject to the normal state. Another 
iteration was then begun by generating a new display depicting the process 
histories advanced 10 sampling intervals in time as illustrated by Figure 2. 

(The dashed vertical lines indicated to the subject the point at which he 

last responded to each process. ) j 

! 

! 
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I 
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Figure 2. An Updated Display 
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each other. 


Before each trial, the subject was told the average number of new events 
h. could exi.'Oot to occur per iteration. He wee not given any iSformuS 
regarding the dynamics of the processes, but was told that he could expect 

also 6 nntT aimn! l r characteristics when operating normally. He 

an 80 not told what parameter changes defined events, but was told that 

hioon? VentS W0Uld generally exhibit similar characteristics, and all would 

iterati^ r6 of‘ P1 h 0n ? U, I Ced K a o ^ passed ‘ The subject was given several 
^ ra training before each trial during which solid vertical lines 

ha^oecurrecL ° 0 ^ pr0Cess histories to mark exactly when and where events 


a ?^i n fi each trial * the subject was asked to keep a log in which he 
described his strategies for event detection and noted characteristics of the 
re. ss ^asurements he used in his attempts to detect events. After each 
trial, he was asked to order these characteristics in terms of their 
usefulness in event detection. r 


APPLICATION OF THE MODEL TO THE EXPERIMENTAL SITUATION 


The event detection model suggests that the human operator in the 
experimental situation just described extracts various features from his 
w“oh rVa “f, 3 ° f , the process measurements. Be attempts to select futures 
which characterize the presence or absence of task related events. Through 

Wi ? 1 th ® P rocesses » the operator has formed estimates of the 
fdn f tion ^efficients with which to combine the features to 
mm* a a . di w scr . iffllmnt faction score. He has also formed estimates of the 
of this score over observations of events and over the 

In observations. The operator generates the likelihood ratio that 

hi ooourred based on the value of the discriminant function score, 

his estimates of the means and variances of the soore, and his estimate of 

Jatio lith a P th™hl^ y occurrence - compares the likelihood 

atio with a threshold that is based solely on the values of oorreot 

responses and the costs of incorrect responses and responds "event" if the 
likelihood ratio exceeds the threshold. 1 W,e 

Four features of the process measurements were seleoted for use with the 
•vont detection model. Selection of tn.ae feature* Z guided by the 
comments of the experimental subjects regarding the characteristics of the 
process measurements they found useful in event detection. The first feature 

of nV °th 1 r m ^ 6raa8n J tUde ° han8eS b6tween 3UCCC38ive measurements in a seouelle 

reversal in Tirlori^^TfhV Th ® SeC ° nd , feature Evolves the presenoe of 
«« direction in this sequence (changes fretn ; ositive slope to 

fh f at .i Ve ’ r / ic l u ver8a ’ of the line se ®»ents connecting the measurements of 
1*1 J a hird feature tests the simultaneous ocourenoe of 

large magnitude ohanges and reversals. The fourth feature, like the first 
is a measure of magnitude changes, but it is much more local in that it 
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1 " TOlV “ °" ly th ' f0Ur "° at »«* measurements of a. proo .„ output . 

of orocw, measurements, th. 

values over earlier measurements. The ueTuht"^*'’ 1 " 1 “ l0r * heavUy tha " 9>» 
the age of the measurement and the rie of ttlt if’*"**. Mponentlalljr u .th 
The value of the first feature” for e «J?. <teCrMM ls * free • 

changes between successive measiremente in » meaaUr * of th « "»gn«tudo 
measurements cf a process- ou^t, iS g^ven b y ^ " °° st recenfc 


a, = ( £ lt(W1 )-2(k) I -exp[- 0 (B-t-tO])/ £, lpt _ 

k= \ 


( 3 ) 


measurement, and 0 is ^e^reT^aram” S€quf!nce » is the most recent 
the feature's value over recent a’nd ^Xr^ 

+~?SiS£*? f n ~ **&« *,™ * 3 P — at a 
measurements over which the features are ealoui^L S6q f nce of P^««s 
measurement displayed for the prooess ends with the last 

used in extracting the features P from the VeT*^ * The cutoff length 

of the features over process measurem^ntf is a free P apa ®®t®r. Values 
not calculated (or, effectively ta J en earlier than the cutoff are 

responded -event'' to theprocessS i ? n J , weight) ’ Xf the subject 

features are calculated over only those foXlowin ? the cutoff, then 

response. The information on the state of ^71®° °°° urrin * after this 
when he responds to the process motivates thi* iU»J °? 88 4 the sub J* ct S8ins 
in the normal state, then on succeeding itJ^^^ X>n8t ^^! lnt, If th * P roc ® a ® is 
an event has occurred, it mustUa^e oe£ur«d m?** the u 3Ub J eofc “nows that if 
the process is in the abnormal state then thi° Wln * hi8 laat re ®P° n9 ®* If 
when the subject keys in his response * On atio/lLt/H*' 00 ^. 8 i8 to normal 

knows that if another event ba* ooournd i?! iterations the subject 
occurred following his last response In * pr f MS ' it must have 

modei) should calculate features only over th * 8Ubjeot (and the 

subject’s last response. ly measurements occurring after the 

representation of normal * ^and* ^bnor mal 0t ^ coeffici ® n ts requires a 
representation was formed using the nr^L pr ^ a measurements. This 

into* 0 , °" “* thlrd t^;T, r ^h\V l J , , 1 ” PUyed * th ' 

into two groups of sequences - normal t Stories are separated 

measurements beginning when a process was rat*.?.! ® bnorma l* Sequences of 
ending when an event ooourred P are defined^^^K tC the norBa l state and 
measurements beginning when an event occurred and «ndi nor “ a1 ' Sequences of 
returned to the normal state are defined to h« nd »^ ndin ? when the P roc ® a ® was 
four features were caJculated over^ J he valw ® ®f the 
in the two groups. A disorinimnt anaivsi* 0 ” 8 ^ ° k u 8ach of 0,0 eequenoea 

differentiate it— t^W. T°n. Sj 
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resulting discriminant function scores for the sequences in each of the two 
groups was also calculated. 

The final requirements for application of the event detection model to 
the experimental situation are estimates of the a priori probabilities of 
event occurrences and the selection of a threshold against whioh likelihood 
ratios of events can be compared. For experimental trials in which one event 
was scheduled to occur per update of the display over the nine processes 
monitored by the subject the a priori probability of an event occurrence in 
each process was fixed at 1/9. For trials in which two events were soheduled 
to occur per display update, the a priori probability was fixed at 2/9. The 
threshold against whioh the likelihood ratios of events are compared is 
assumed to remain constant through an experimental trial. The magnitude of 
this constant is a free parameter. 


RESULTS 


Figure 3 compares the event detection performance of t. e model with the 
actual performance of each of the eight subjects in the third trial of the 
experiment. In this trial, 20 events were scheduled to occur in the trials 
given subjects A,B,C, and D, while 40 events were scheduled to occur in the 
trials given subjects E,F,0, and K. In applying the model to each of these 
trials, the nymber of measurements over which features were extracted (cutoff 



length) and the relative weighting of recent and older points (0) were 
adjusted to improve the fit of the model’s performance to each subject’s 
performance. The value of the threshold against which likelihood ratios of 
events were compared was also adjusted to improve the fit. Figure 3 reveals 
a high degree of correspondence between the model's performance and the 
performance of most subjects. 

Figure 4 compares the event detection performance of the model with the 
actual performance of the eight subjects in the second trial of the 
experiment. In this trial, 40 events were scheduled to occur in the trials 
given subjects A,B,C, and D, while 20 events were scheduled to occur in the 
trials given subjects E,P,G, and H. In applying the model to eroh of these 
trials, none of the parameters of the model were changed from the settings 
used to obtain the results presented in Figure 3. Despite the fact that the 
numbers of events scheduled in these trials differ from those in the trials 
used to assign the values of the parameters, the correspondence between the 
model's performance and the performance of most subjects is reasonable. 



Table 1 compares the mean detection times (in terms of the number of 
sampling Intervals which elapsed from the occurrence of an event to the time 
of its detection) for hits common to both subject and model in the trials 
presented in Figures 3 and 4 • It should be noted that the fact that the mean 
detection times of the model are consistently smaller than those of the 
subjects is an artifact of the manner in which the model's performance was 
investigated. The model was tested on the process histories displayed to s 
subject in his experimental trial. In these trials, a process was returned 
to the normal state at the point at which the subject detected an event in 
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the process. Ihus, in going over the process histories the model can never 
respond to an event later than the subject responded' to it. If the model 
fails to respond to an event by the time of the subject's response, the model 
is scored as havipg missed that event. 

Table 1 . - Comparison of Mean Detection Times 



Trial 

1 

Trial 

2 

Subject 

Code 

Subject 

Model 

Subject 

Model 

A 

24 

19 

24 

20 

E 

26 

21 

25 

19 

C 

28 

17 

28 

17 

D 

32 

21 

30 

16 

E 

18 

13 

22 

. 17 

F 

17 

14 

27 

16 

G 

20 

17 

27 

20 

K 

20 

17 

22 

18 


Ue plan to evaluate the model in the near future using a somewhat 
different approach. Rather than running the model over the process histories 
displayed to a subject on an earlier experimental trial (and constraining the 
timing of the model's responses by the timing of the subject's responses in 
that trial), we will use the model in place of the subject in the event 
detection experiment. Processes in which events occur will then remain in 
the abnormal state until the model responds to the process The only 
constraint on the model’s detection times will be the end of the experimental 
trial. Because the model's detection time for each event need no longer be 
less than or equal to the subject's detection time for that event, we expect 
that, for a given number of hits, the model's threshold can be raised to 
achieve the longer mean detection times and smaller numbers of false alarms 
characteristic of the subjects in the experiment. 


CONCLUDING REMARKS 


In applying the event detection model to the experimental situation 
described in this paper, we studied a situation in which the subject was 
forced to respond yes or no to the possibility of an event related to each of 
nine processes. In general, the human operator is not forced to make such 
yes/no decisions with respeot to eaoh of his tasks. Instead he uses his 
estimates of the probabilities of task-related events (which the event 
detection model generates) in deciding how to allocate his attention among 
his tasks. We plan to run an experiment investigating the human's attention 
allocation performance in a multiple process monitoring situation similar to 
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